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SUMMARY 



] ^ Two computer programs >K^re wxi^tten tp provide ^n-line aiding to ' 

• » - 

human problem solv6rs. >B6th* programs were.written in tima-shared BASIC, 
and were .designed for "membership" problems. In this kind^of p^oBlem, 
there are several English sentences and imp1icit-in the sentences are 
various .relations; the task is' to Jnfir a ijiembership structure that is 
compatible with all^ the logical constraints. Membership problems may be 



cast in various settings, such as a muVder mystery where a culprit is to 
be identified/. 

* One program (FIRSf) was based on Findler's "Universal Puzzle Solver" 

concept; the oth-er {(^BE') used Wang's theorem-prover logic. In both pro.- 

^ grams, the human operator .converted English problem, sentences tp lo^ica] 

• membership" relations. The programs kept 'track of all .reiatilDns entered, 

indicated when more data inputs were needed, and scored whether a correct 

*' * * ' * , 

answer was achieved. 



Of the two programs, FIRST appears to be most feasible with ordinary 

'college subjects, ft accepts logical, inputs- in a, near-English format, and 

shows current logical' status of a problem via tabular arrays of X*s-and O's 

• The present vefsion of GABE. used a strict "p, q, r" logical notation; 

college subjects fi^nd this..difficul t and unsatisfactory. 

The str'ucture of the.flRSt program suggests a "depth-of.-inference" 

measurement technique. ,When>all possible logical paths in a membershijy- 

.problem are known, th^*'-" depth" 0/ any given node in the path can be obtai'ne 

frorti pnobabil ity-of-success. numbers .at that no4e; also it appears that a ^ 

subject '^s Hogical^^progress .alQjng a' pattt can be^computed and displayed. * 

Further emp|,rical work-wijl explore the usefulness of such depth measures 

« - 

^for scoring individual pejcformances, and for teaching, problem-sol ving 
heuristics in technical materials, ^ * , . 
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I. 'INTRODUCTION 



Certain intellectual tasks, such 'as estimating anfi combining prob- 
abilitV information, controlling several aircraft, or troubleshooting < 

r • ' , . ; . ; • . , 

•equipment, may be aided by a special^* type of computer' program. ^This 
type of prQgram\has in it a representation lof the real-world setting, 
and can quickly perform the library, bookkeeping, and calculatifi^ chores; ' 
the controlling humani remains on line, contributing inputs and^^udgments.^ 
^It -is possible to achieve a genuine man-computer j'nteracti on in this way; 
^ and the output may be appreCriabl-y ^ett^ than either' man or computer 

could produce alone. This report describes some preliminary investigations, 

^ . .> 1 . 

of computer-program aids for humans who are attempting to solve verbal 
, >«. > 

« • ' »* 

problems and verbal puzzles. ' ' ^ 

Motivation foK selecting verbal problems^for our 'attention c'ame frpm 
several places. A practical reason for building such aids- derives from 
the fact that some of the hardest problems facing humans are cast in part- 
verbal form. A familiar example here-is the technician -who must operate^, 
calibrate, or troubleshoot a complicated electronic or mechanical-sdevice. 
His tech manuals, diagrams, iind previous training may sometimes provide 
adequate information for |iim. But his performance must be a mixture of " 

hypothesis-formation-and-test behaviors, combrtned witii inferences about 

« 

the meaning .of observed event?. When he talks and ttiinks about his actions, 
the technician is apt to use qualitative* verbal models of the physical 
actions in the equipment. His seqaences of checks may be remembered in 
verbal form. Furthermore*, his attempts to validate his i interpretations 
may be confirmed, controlled, contradicted, or frustrated^ by verbal sen- 
tences in tech manuals. Concjs^ivably, ^ general software aid could assist 



the human in " understand ifrg" technical informatio.n, in "keepin^^:fnngs ^ 

straight," deciding- the ''?ls't thing to do. now, .knowing what- tests I ^ave 
• . • . f - * 

done" so far,'^^nd in ivoiding "doing the same thing, over and Gver ,again.\ 

Many studies show that tec+inic'ians are- 'usually "redundant and non-optimal 

in their .search behavior,- Qven thoujh the correct information may reside 

in documentary .soirees.-, Even though "it is all in the tech manual," the ^ * 

.search process may be quite ineffective. People must be taught, and ^ 

taught specifically^ on ways to extract, information from.gompley sentences. 

We ^expect that a ^ystematio investigation of verbal _ problem-sol ving 

processes would serve to pinpoint :iust where the psychojogical difficulties 

aref Verbal problems •usually rest upon a' definite underlyijig structure. • 

■ftiis structure has to be inferred" from English words, and elements of the 

•v ^ ■ 

•struct1tt=e-are then operated upon by the application of logical processes., 

\ ' . . . ' . 

If a comiiuter program requires* the human solver to entjB4r:,.tlie., essential . . 
relations in^a problem*, then Vc^ program would always know, just which of 
these delations are- not yet .realized bythe person attempting the,problem. 
Th^e program qould^show the solver what his present solution status+is, and 

• • • • * 

just where the remaining loaical gaps are. In fact, as we. indicate later, < 
this approach leads to a ^way of mea^urf^jptie depth of inference required 
in a given verbal-problem. ... 

-'Finally, the investigation of verbal problem-solving relates to other 
research at the Behavioral Technology t-aboratories', concerned with' the '^r 

• ' * * f \ 

analysis 6f text processing. Verbally stated probl«ins, of the sort- sfudiei* 
here, are useful for studying;- iritersentence processing, in distinction to < 
the intrasentence process i rig that ha? been the almost exclusive concern of 
traditional readind research. The importance of-. understanding more about 



intersentence processing lies in its contributions to the compnefrenSion 
of text passives; (*1) its saliency for ^urf8erstanding xlifferent types 
of texts, a tqBic^;:^^^^^^ "P^^ attention from the theorfst^, whd 



have tended ito restrict Ifeir studies to simple narrative/forms;X (2) its 
potentia-l as B rich% source^f ihformatiofP about hfgher level cognitive* 
•prpgesses, and (3) the relevance o/ the information-processing, sKi lis * 
it requires to effective reading. Many^ these issued are discuss.ed in 
greateV detail by Ri^Rey. (1977). ' ^ 

'There is reason to beVieve that integration of inforrtiation atrpss 
sentences may require different kinds of cognitive processes than those 
required by intrasenleoce processing. This ^assumption* is based on obser-, 
vatiori^ in our; laboratory of two forms df what we call decoupled reading. 
In one form, the reader decides to read'the passage/ but nof to read for 
comprehension. In the other. form, the ii>eader's intention js^ to jread for. 
comprehehsion,. but somewhere in the passage he realizes he does not re-^ 



member the raeanitrg of any* of the last few sentences^ He had^been reading, 
at a word-by-word level ,. and had the feeling *hat he. umJerstooil^what *he 

read, but he suddenly realizes that the focus of his attention was occupied 

*^ • ' ' - ' 

wjth something- else. * ' • - , . ' 

Word problems shoul^l'b;^ useful for triyestigating , these f^igher level, 

integrative processes, since these problems are easily read sentence-by- 

sentence, but cannot t?e solved v^thout-a. lirge amount jDf more difficult 

^ — ' - • *^ 

intersentence processing. that-entails deeper levels of inference. The 
whole question of what intersentence relationships influence cognitive 
proc'e^ses mediating comprehension and^ memory^, des^v^s^iirtfensi ve' investi- 
gation. Some initial work has been done on story granmars for narrative 



forms (Rumelhart, 1975; Thorndyke, 1^77; Mandler and Johnson, 1977), ^ 

but- this is just a beginning/^ Using this research* as a point of depart- 

ure, work is in process at our laboratory- on a second generation text - 

granwar that wilV encompass forms other than the narrative form (Gordon, 

Munro, and Rigney, in press). 

. One- way' to characterize the*text structure problem'is as follows. 

Suppose that; eJcactly the s^me words were arranged in five different v/ays; 

(1) as a random string of characters, (2) as a random string^of words, 

(^) as a random list of sentences, (4) as a conventional piaragraph with 

topic sentence and amplifying sentences, and, finally, (5) as a ward 

puzzle. If these five different arrangements of characters were given 

to subjects to read, clearly each arrangement would evoke different kinds 

of cognitive processing., under the same objective. If subjects were given 

the objective of memorizing the passages, there would-be differences among 

them in time to completion, errors in protocols, an^s^^^ngth of retention. 

If subjects. were told to read the pas'sages for comprehension,^ there also 
. . .1 

^wouTd be^ differences among the dependent variables. It would. In fact, 

be difficult to find common measures of comprehension. The meaning of 
• • . /v ' ' 

e&cK passage ,'would be quite different. Why? 

t • * 

* Our interest is tn the different answers to this question required 
for different text forms ab9ve C3), the rSindom list of sentences. We do 



^ text Torms aDQve tne random nst or sentences, 
not know, at this point, how many meta-sentence level forms ,exi§t. 



Possibly there are many^clas^s and many v^iations within each class. 

ft 

Rigney (1976) speculated that there are at least four; narrative, explana- 
tion, description, and prescription. It remains to be seen whether this 
will b^ a useful classification. We are reminded of some interesting 
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variations in text forms. For example, in the Bransford .and Johfson . 
1^(1973) passage on 'washing cTothes, the meaning of the entire passage 
depends on the^formatiOn that it is, about washing clothes. Each 
sentence 1n the passage relates to. washing^lothes. This seems to be , 
the crucial intersentence relationship. Other intersentence relations 
seem to be primarily- thbse found in preseriptionss *var-ious i)bjejt^^^ 
manipulated in temporal sequence," determined at leas>t partly by causal 
relationsh-ips. But the sentences in the passage are so worded that the 
prescription might be for any of a number of .tasks, 'which leaves the - 
reader confused until the information is given him that the passage is» 
about washing clothes. Bransford andT Johnson demonstrated that subjects 
given this infgrmation before the passage was read had" higher comprehen- 
sion and recall scores than subjects who did not have this -prior infor- 
mation. ' / . ^ . . 

Word problems embody a different text form. The first sentence 
establishes a cast of characters and some of their attributes, the 
following sentences clesc^be relationships among some of these attributes 
without .identifying which characters are involved. The last seritence^is 
a question requiring the identification of the charact^ with a specified 
attributfe. This r^equires the reader to (1) do deeper processing of his 
prior knowledge, (2) to make inferences about which cl>aracter could ^ 
'possess which attribute, and (3) to hold a large amount of infprmation 
ih temporary store. , An example^ of a word problem i^s: . 

Mr.-^Scatt, his sister, his son, and his daughter 
^are tennis players. . . * 

The best player's twin and the worst player are 
of the opposite sex. 
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. The best player and the worst player are 

the same agie. ^ ' , ' 

Who is the best player?, 
^ Solving 'this problem requires deeper processing of a kinships schema 
(Monro ^ahd Rigneyj 1977) to retrieve the information that Scott's sister 
could be the same age as hjs: ihildren, and to make 'j,nferences that can be 
formalized irv the propositional calculus. These .inferences- also are 

••■'/. ■ • ^ ^ 

deeper than a reader ordinarily would indulge in if the last question was 

omitted. ' 

" ' , * ^ ^* 

*We .view this kind of. text form as being useful for learning more about 

how people do deeper, processing and deeper.inference dur^g inter- 

^entence processing, and for a measure of current information processing , 

.papafcityi Hunt' s^ (1977) XIP capacity^, usjng . text processing skills , rather 

^■^^ — 

' ' * 

than the simple tasks of^the verbbl: learning laboratory that theorists 

^ ^ . \ * ' - . . . , 

presume to be involved in text processing but that have not been demon- 

stratQd to underly the tasks of intersentence processing. 

" The principal thrusts of tha research- described here were an^explora- * 

' ^ -^-^ . < 

tory investigation of the difficulty of word problems /or students, and 
an investigation of how students interact with a computer program designed 
to accept their inferences during intersentence processing and to give 
them feedback that would assist them in solving word problems. 

To date, we have tried out two interactive computer programs that 

, might be expected to serve as^ probl em- solving aids. These programs have 

> 

not yet been fully evaluated; but they^^re now working, they do "solve" 
verbal problems, and we have gained Isome experience with college students 
*using them on line. In this report, we give a simple example of a word 



problem and its solution. Then we describe the computer programs them- 
selves; The program listings along with sample problem solutions iire ' 
prijited in full in the Appendix. The last part of the report recapit- 
ulates our experiences with the programs so far, and offers some sugges 
tionV^'for extending these investigations. • ^ 
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vlj an example of a word problem and its 
■ • solution 



To fix^'the present setting, let us turn ta^the following reference i 

problem, which was originated some tiecades ago by the ^nglish puzzle ext>ert 

Henry Dudeney, and^ i§ ^presented here in Americanized i/orm. 

^ 1. Smith, Jones and Robinson are the engineer, brakeman and 
fireman on a train, but not necessarily in that order. 
Riding the train -are three passengers with the same three 
surnames, to be identified in the following premises by 
a "Mr, \ before their names. 

Mr. .Robinson lives in Los j^ngeles. 

The brakeman lives in Omaha. 

Mr. Jones Ipng ago/forgot all the algebra he learned in 
high school,, . / : 

Thfe passenger whose name is the -^same as the brakeman 's 
(ives in Chrcago. ^ / 

The brakeman and one of the passengers, a distinguished 
mathematical physicist, attend the sa(me church; 




5. 
6. 
7. 



Smith beat the fireman at billiards. 

Who is the lengineer?^" ' 

This is a class-membership problem. When well-formed, such problems 

have a unique solution, the reasoning can beifollowed by ordinary people, 

and tjje special information demands are not. excessive. Thus we suppose ^ 

that everybody kn^s that Chicago, Omaha and Los. Angeles are cities; and 

everybody also knows that if Smith beat the' f ireman\at billiards, as stated 

♦ 

in premise 7, then Smith cannot be the fireman. 

' When educated adults are given this problem without^aids or without 
any special training, -they get the right answer withi« 15 minutes or so 
(about 80% of one large psychology class solved it). A few, perhaps 



five percent,' will not seriously attempt to solve it ("I'm not'good at . 
this sort of thing"); some will approach the problem in a proper spirit,- 
but will make mistakes and come up with a wrong answer; a very few wilT 
propose answers to the problem on some non-logical grounds .("Physicists 



. just don't live iij Omaha, they'd 1be moreTTikely to live in L.A/or ., 
Xhicago")r "SffetlgSTfOT sblVers show maj'fed-individual differences in ^ 
their solution time (some get it in less than two minutesj; the subjects 
will also differ in their confidence about their -reasoning processes* 
After lieing stiown a ^^logically sound path tb/a solution, however, very • 

few educated adults will doubt the answer. . • . • , 

^ 

It may be helpful to set ,up a tabular, .representation .^f the problem; 

' : . ^ ^ ' 

"^in Figure 1, the matrix on^the left has to do with the railroad gwpfloyees 
and the right-hand niatr^ix concerns the passengers. When a logical possi- 
bility is eliminated* we put^an "X" in a eelH when a eel} 1 s trtte^j^we-- , 
insert a smalT do^t. ' .--^v^ - , 
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Figure 1. Two Matrices for the "Smith-Oones-Robinson'! 
Problem. 
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Right off, we can enter a dot for the lower left-hand cornef of 
the second matrix: from premise 1, Mr. Bobinson lives in Los Angeles, 
•and not in the other two cities. So there must also be X's in the 



l^obinson cells for Omaha ^nd Ch^ago; and X*s in the tos Angeles column- 
for Mr. Smith ancj^Mr. Jonesy^As We have already noticed, premise 7 
plainly indicates that Spnth is riot the fireman, so we enter an X in 
the appropriate plac^n the left-hand matrix. Now the tabli^looks ^ 
like this: 
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Mr. Smith » 
Mr. Jones 
Mr. Robinson 



X » X 



. Figure 2. "Smith-Jones-Robinson" Matrices, after 
Data are Entered from Premises ,1 and 7, 



There are' still a dozen indeterminate cells in the two tabbies, so 

V 

we must now begin to combine information from two or more sentences. * - 
Scanning the set, we see that premises 3 and 6 imply that the physicist 
lives in Omaha; and, since we already k.now he cannot be Mr. Robinson, then 
he must be eitjier Mr. Jones or Mr. Smith.* But from premise 4, the physi- 
cist cannot be Mr. Jones (because you cannot be a physicist and still have 
forgotten'all ydur high school algebra)*. Hence, when you take 3, 4, and 
Sotogether, you see that the physicist must be Mr. Smith. This effectively 
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' ' fills in the second table, because there is nowhere left for Mr.. Jones 
to> live except in Chicago. Now we can go back to the first table, and 
s6e that from premise 5, Jones must be the brakesian; and so the final 
jinswer is tha't Smith is. the engineer. The problem is solved. .OjLiJ.ourse, 
- this fTarticular problem can be solved without any computers; or perhaps 
without any graphs or record^fng techniques. For problems that are__ 
longer or that are more complicated; thougTi, the potential usefulness 
bf computer-4iding increases. It might even be possible to'teach sub- 
jects, via computer-aiding, to become champion solvers of this kind of . - 
probl^. ' 




X 
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III. THE COMPUTER' PROGRAMS 

To our knowledge, tliere are* two^publ jsted, r.eporJ:s.-oa. computer-i^ 
-^^-JSiVysteras ,for solv-ing-pV^oblems like the Smith-Jones-RobinSon example. In 
1956, John G. Kemeny programmed -a gigantic twelve-premise problem which | 
Lewis Carroll had posed about 80 -years earlier. Twenty years^ ago, his • 
solutioft' took four minutfes on an IBM 704 (Kemehy^ 1956), a complete 
printing of the "truth table"' of the problem would hSiVe taken 13 hours! 
"(With present technology, the computations *would have taken several seconds, 
/and the ^printing some few minutes)'.. ' * \ 

Seventeen years after Kemeny's tour de force -at RAND, Nicholas Findler 
(1973)/ described a "Universal Puzzle Solver" program. Findler'sTprogram, 
which was written in SNOBOj-^ operated via a membership Idgic structure and 
* a-recursive searcfr^broutine. English words for set members and relations 
have to be entered^ along with absolute and conditional membership'state-"*^^ 
ments; these logical statements are derived, by a*human, from tfte original. 
English-language problem sentences. Once all the prqblem and igWution ^ 
^ "Cipndititfns are entered, the program sets up appropriate arrSy^s*, and then 
searches ^these for a *$olution. The starch is systematic But brute ^Force. 
; On medium- fast processors,^ a problem runs i-h a second or two|. . Answer^ 
are printed in constrained English sentienc^es.^ T4ie progra^n hai elegaat 
provisions for multi-stage problems, for conditional relations between 
variables, and for" "output of results. A most unusual feaiure of Find^er's 
work concerns the generaility of the program; at the end of his paper 
describing the program, he says^ he ^cannot see any way, or any need, to 

,^xtend \ts capabilUies furt^ (Findler, 1973). Because of thi6 gener- - 

,v • ^ 3 , - , " ^ 

ality, and the ingenuity of F.indler's search routine, we decitled to adapt 



Findler's concept for one of our aiding progr^ams;^ we oalled our version 
FIRST (Findler Interactive Routine forjub^'ect Tr.aifi^). Because t;me- 
share SNOBOL was not available on ourC mini -computer', we jrecoded p^arts of 
Findler!s progr-am into extended .BASIC. Al'So, severVI features were adde 
to -suit our purposes better: for instance, "provision was made far identi- 
fication and correction of errors ^ entering problem information/, the * 
subject's information state was .tracked at >fi^r^ step; tabular graphs of " 
logical inclusions and exclusions already achieved were , avail able on demand 

The -first important inputs to FIRST from the huipan subject are dimen- 
^sion and set specifications. In the San FrariiJ^sco prot^lem sho^n in the 
*"Appendix, there are five 4)eople who get to worl^f^j^itve^diffefent "^vfays; 
there are thqn, five members in set ly(Al, ^r^'t^\^\ Davfe ind Ed), and , 
five in set 2 J[bike, car, B^RT, bus^ walk). Nineteen sentences^ dire listed, 
and these sentencei contain enough information to'alloW each person te- be 



assoGiated with a mode of transportation. Menjbjer^fiip relations from these 
sentences are^writtfen in "CON'^^^^j>^NO?-CON" form. "^Or* meari\^a strict 
logical connective is established,-^ "NOT-CQN" i3 a logical exclusion. So 
when sentence 14 says "Dave greets his driver with'Jgood morn-ing'' everyday, 

a solver might enter th€ following FIRST statemenl 

^ ' ^ ^\ 

DAVE, NOT- CON, WALKING ^ 
DAVE, NOT-CON, BIKE - • • 

DATE, NOT-CON, BART ' \* " . 

Some Vocal information is needed in this problem*. Tbe -logical assertion 
about BART is. less obvious than the other -two; you ha>te to kr)6w that BART 
is a rail transit system^ and also that a BART driver is inaccessible, and 
cannot be spoken to (the^driver xloesn ' t "drive" the v<ehicle, a computer 
drives it; the driver is there for override. purposes).. ^ 



pv ^ , * fx 

When fthe subject working a problem wants a "present-status" prlntoiiJ:, ^ 
he hits a cootrorkey, and a tabuT^jp-^ijr^sentation appears on ti<fe terminal;" 
this table shows "0" for membership and "X"* for non-membBrship. * In a five- 
Variable problem,! if there were three X's in, a giver) column, then the ^ 
solvfif, might focus on that, variable,* and go over the problem sentences again, 
in order to find a 'fourth exclusion and thus pin down the identity of the ' 
column member. ' . 

We selected Wang's theorem- pro ver system, called GABE ir> the Appendix, 
as the model for the second program. As in the Findler approach , to' use 
the system a human has to accomplish some trans latiorf of complex English 
sentences into logical relations^ using only tiie'"and," "ot\" and "not" 
operators. But instead of a Findler-style recursive §earch for one or a 
few- right answers, in^Wahg's system, after you have inserted the premises; 
you *then must ask the program whether a given outcome statement is valid 
or not* Thrus, after coding the Smith- Jones-Robinson problem into W&ng 
notation,' you "Would have to suggest to the program the following three 
"theorems:" ; " . . 

"'^ . ^mith is the engineer. 

Jo'nes. ts the engineer. ' 
r Romnson is the engineer. 

^ \ • 

All three theorems would be "tested^" via the Wang algorithm; of course, 
only the first would turn 'out to be valid, if the problem relations were 
properly entered. As it is now set up, the system does not list all • . 
possible valid statements from a set of premises; you have to ^isk it about 
specific oties that are of Interest. In fact, from a small but fairfy rich 
set of preiTjises, an enormous number of valid "theorems" can be derived, xind 
;it would often be impractical to print the' whole, list. ' - ^ • 
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^'.Putting -a verbal problem' 'Vnto the Wang process is more abstract 

C 

to .the subject than fIRST. English problem statements are converted 

into bare representations; then, terms in these representations are given. 

a symbolic translation into logical operators*. • As -an illustration, we ... 

tajce^the "murcieri^ problem from Raphael (1976). ' ] ' 

• • ' "* 

. Wang's algorithm worlds by fallowing a'staged reduction r9Utifle. The 

procedure writes' down ^ series of logical liness each simpler than the 

preceding one.^ The simplification, continues until fhe same logical express- 

ion occurs pn both sides of a centfal arrow, ►'or until a mismatclh occurs. 

sfhe Appendix '^hows this line-shortening^ process as it wo>ki>d- ir> on^ problem. 

We origiaally hoped that human subjects could le§rn^^^ imitating the a-lgo- 

rithm, hqw to process logical terms; or at least,^we thought that some, 

subjects would become intrigued with Wang's reduction and proof Scheife. 

This view was naive^as it turned oift; the details of the Wang. operation ^ 

are totally myster/ous, and also totally unint.eresting, to the ordinary 

adult. . ' , • * 




THE PROBLEM 



'The Facts • • 

The maid said that she saw the butler in the 'living r5oft*» The ^living * 

, room adjoins the kitchen. The shot was fired in the kitchen and could 

. be heard in all nearby rooms. The butler, who has good hearing, said 

he did not hear the shot- 



To Prove 

If the maid told the truth, ,the butter lied. ^ 



r 



THE. REPRESENTATION 



-5P' = 
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r = 
= 
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The maid told the truth 

The butler was in . the living room 

Tf* butler wa Senear the kitchen 

The butler heard the shot 

The butler told the tnuth 
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MEANING 
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•(If the. maid told the truth, the^'but^er 

was in the living room). 
. (If the butler was in the living room, 
h? was near the kitchen)^ 
(If he was near the kitchen, he heard 
the shot). ^ 
pu-v - s '(If he told the truth, he did. not hear 
^the shot). * . 



MP V q 



pq V r 
prVs 



THEOREM 



pu 



pp V - u (If the. maid told the truth, the butler 
did not)., ^ 



Figure 3. A Mystery Solvea by Prepositional Calculus. 
Tlie Rroblqm and its representation. 
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"IV. TRYOUT j)F PROGRAMS WITH HUMAN . ' ' 
■f , . • ■ -V ^ SUBJECTS • ' ^ 

When materials like the two aiding programs- described above are , 
prepared, there are some questions^ which caiV4)e answered only, by experi- 
mentation. For ins-tance, do the. programs teach problem-solving mere 
■-^^^WffectTve^TtharF does simple undirected practice; W^ll^ subjects attempt 
to imivtat^' the~computfe"f~way of doing things*?^ -After a^few problem -sessions* 
on computer ^terminal, what are the transfer 'effegts firom one problem . 
to another? Another clas? of issues' concerns feasibility of^e so/tware.- 
concept. >Can ordinary people use the programrand v^i 1 1° they readily usfe . 
•it? Are the materials self-administering and easy to run? Without stanci-^ 
by progranmer staff; do subjects seem comfortable in the situation? Does 
- performance seem to, improve? What kind erf performance niodel do the sub- 
jects' appear to follow, etc. 'It is to this secomj-<^lass of questlB-ns *' ;. ; 
th^ -this part of the report is addressed; the experiences reported 
■ here' are based on a grab sample of California State. University under- 
graduates. Our impressions so. far can be imparted'quickly, under half 
a dozen headings. = ' > - . • " . 

: • 

• 1 . General feasibility . Both programs run at present on a time- 

•'"^ shared PDP-n. They probably will run on any ipedium- capacity time-shared 

system. No remarkable operating problems arose iTi»ordinary program us^ 
■ ■ . • .■ . '■ ■ * ' • * ^ 

-though we often wished we had a better restart procedure. "NaJAher pro, 

*gram had any provision for referring to a library ofi^jroblems , so to 'start 

each problem, a staff member usually handed the subject the problem sen- • 

^ ' tences on a\separate^-piece'^of , p'iiper, and stayed nearby while the subject 

worked.' sJcause of the large amount of text material, and because the 
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subjects often wanted to refer to some previous logic table or data • 
entry, ft was necessary to employ' hard-copy teletype tejjminals; video 
terminals could be used only for small problems. Some "large problems 
took two or, three feet of paper to reach a solution. One incidental 
result; with an assistant nearby, subjects were often tempted to engage 
(the assistant in" conversation about the problem sentences, and to seek 
some immediate confirmation of the logical expressions being entered 
into the terminal. ^ 

For Wang*s reduction program, GABE, it was not feasible 'for ordinary* 
students to convert English sentences into logical symbols. This was 
probably due to the general lack of fluency with the, logical' operator 
notation:" p, q, p , a , v , = , etc. Also, there were often two stages 
of "stripping" the English sentence down into symbols.; We tried to give 
a "short course" in the notation to several people, but there was general 
and specific resistance: generally against anj^ logical symbolism, and 
specifically against the ( p p v q) representation of if-then or impli-^ 
cation. We conclude that any serious use_j)f the Wang/'eduction concept 
as an aid would require considerable pre-requisite training in logical > 
notation and in translation. We suppose, too, that people who are fluent 
in artificial languages, such as computer programmers, woufd find the sys 
tern more acceptable. 

2. Data Input .. There is no doubt that subjects find the teletype 
format to be a "slowdown," and somewhat' frustrating. The presentation is 
"all .words," and constrained words at that;. everything has to be typed in 
^nd on a fairly large problem; the subject cannot be sure v/hether or not 
he/she has'^enough data to reach a solution. He then must request the 
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computer to print out a current state table (or the program ifsejf 
decides to print one); and at ordinary teletype speeds.' this tates some ^ 
time and interrupts the solution process. So the clanking termirtal may 
be a. real- distraction to the solver. There are inputroutput devices^on . 
the horizon that could help to alleviate this problem, v v 

3. Individual DifferencesT . For entering logic from plain, des- 
criptive..sentences, the FIRST aiding program reduces individual differ- 
ences to near zero. In the first sweep through the problem sentences, 
when each sentence is taken separately, the .human silver simply converts 
the sentence meaning into a' "CON" (membership) statements, or a "NOT-CON" 
(exclusion) statement. "Mr. Robinson CON: Los Angeles," would be one 
example from our reference problem. 

When the subject has to combine information from two or more sentences 
or has to realize some "deeper" aspect of the facts presented, th^n the 
variation between people can be quite marked. If facts from two or more , 
sentences ara processed in such a way as to provide a new, non- trivial 
inference, then ffie subject first has to select the sentences to be con- 
si dered -together; this means that a dimensional scanning operation must 

be performed. Next -the subject has to do further processing to reach a 

h ■ 

neW|^inference. \ 

The combining processes can be illustrated with one of our favorite 
problems, "The Murderer," taken from Summers (1968): 

Murder occurred one evening in the home of a married couple 
and their son and daughter. One member of the family murdered ^ 
another member, the third member witnessed the crime, and the • * , 
fourth member was an accessory after thei'act. 

1. The accessor'y and the witness were of opposite sex, 

2. .The oldest mejnber and the witness fltoere of oppos>fe sex, 

i 3. The' youngest member and the victim were of opposite sex. 
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4. The 'accessory was older than the victim. 

5. The father was the oldest member. ' 

6. The killer was not the youngest member. ' , 

Who was--what? ■ , * ' ' - 

Each of, the first three sentences in thiS problem contains an easy 
conditional" relation: for instance, (1) implies that if the accessory is 
female, then the witness is male, and vice versa.' Anybody who can read 
English will be able to enter these relations into the program. Some of 
the combinations between sentences are easy, too. Look at premises (1) 
afnd (2).,'''The last seven words of these two premises are identical, anc?*"^ 
the sentences are right next to each other; so the circumstances favor 
a comparison between the two. It then quickly appears that the oldest 
* member and the accessory are of the same sex. Other combinations may not 
be quite so easy, but are still likely to be achieved. For example, from 
premise (5) we know that the father was the oldest; so we could already 
infer, at this stage in the search for a solution, that the witness w^is 
felnale. ' ^ , • 

A more difficult, but also moVe intellectually satisfying, inference 
chain ^oes. as follows. Suppose^we explore the identity of the "youngest 
member,", and start working across sentences. From (6) the youngest mem- 

er cannot be the killer; from (3) the youngest member canno't be the yicti 

4 - ' ^ ' 

so the youngest member must be either the .accessory or witness^ But fron 

* ' * ^ ' - / 

|4) we see that the accessary is ^Ider than somebody , ahcf'hence also cafinot 

be the youngest. Therefore th^e youngest must j)e the witness since alVothe 

possibilities have been eliminated, the difficulty in attaining- thi/ chain 

of reasoning stems mostly from the (4) inference about the accessory. Scan 

ning premises (6) and {i) was relatively easy and direct, because/both have 
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straight-forward language mentioning /"youngest member." But in premise 
(4), youngest member does not appear/ as a term per se ; we have to deduce 
something about youngest member from the "older^ than" relation. 

The problem is now. easily solved; the wit^ness is the daughter, the 
accessory is the father, and so oh. There are several other logical 
paths that Can reach a correct solution; or, all possible role-membership 
combinations (24 in this particular problem) could be ^tried and tested , 
against the original problem sentences until an acceptable set of assign- 
ments met the conditions (the original Findler program would actually pro- 
ceed in this manner). / 

We have seen enough soyution attempts to believe that multi -sentence 
scanning, selection, and ccimbining Skills may be the key to successful 
problem-solving of this t/pe. The basic identification and negation logic 
'is apparently easy enouon, once the .appropriate meaning sources are put 
together in a small package of critical phrases, and examined closely for 
their logical implications. If this view. proves to be correct, then effec- 
tive training methods will focus heavily on the cognitive processing of 
.several temporarily-combined sentences or I'ohg phrases, and not on the 
strictly logical processing of identification, negation, and conditionality 

relations. To put it another way: once you are looking at .the right 

I 

phrases and relations to combine, and confine '|your at^ntion to just one 
or two main .inclusions or conditionalities, then the logic itself is easy.. 



4. Depth of Inference .* It appears, th^n, that subjects/who use 

/ 

• our FIRST version (|f Findler's concept are performing a complex trans- 
latidn task. - English sentences ?ire, read, and the logical gist of the 

• sentence(s)" is typed into the terminal using the, "CON" or "NOT-CON" 

• ^ 

entry conventions. Variable names remain in English, and .part of the 
solution output appears'as a simple English sentence. The computer 
a:^ways knows, then, the exact logical relations that the subject has 

^.put into the mac^flne, and the order in which these were entered. It is 
perhaps useful to define depth of inference in terms of (1) the • 

/^ prot?ability that a given inference is ever achieved, in a reference 

sam'ple of subjects, and (2) the primacy wfth which a logical relation 

is deduced. Both probability and primacy Values can be extracted from 

computer records of problem attempts . -j:J*e Murderer example given above 

^ permitted* an easy and convincing decision that the youngest member was 

not the victim, and not' the killer; it was much harder, as we saw immedi- 

, ately to perceive that the youngest member could not be the accessory 

either. The performance of subj,ects could be easily checked, by\coui1ting 

_ . - " • ' 

the frequency and time order of the following three entries into the 
FIRST logical arrays: 

YOUNGEST MEMBER: NOT-CON: VICTIM 
YOUNGEST MEMBER: N0T-C0N4 .KILLER 
YOUNGEST MEMBER: NOT- CON: ACCESSORY 



* We use this phrase instead of the overwprked "depth of processing 
of Craik and Lockhart (1972), which they defined as the- deployment of a 
flexible processor over any of several stages of .processing,- presumed to 
•intervene between sensory inputs and semantic processing in LTM. Depth 
of inference could be considered to be a form of the latter, a^jdthus 
might be one of many kinds of deep processing. 
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Depth-of- inference indexes^ theft, can readily be determined in 
\e computer-aided probleirf-s-jtuation. These could be useful at the 
individual leyfel (what is this subject's' average depth-of- inference 
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in the.first five minutes of some set of reference problems?), or at 
the group performance level (which inferences- in this particular problem 
are deepest?). ^Obv^ously, depth-ofrinference indicaiors could be used '| 
to check the effectiveness of a training program, or oi some other . j 
intervention. When .properly standardized, problem inferences could be 
scored for depth, and individuals ranked according to their performance/ 

The logical inference task, we expect, requiiqes soma elaborative 
processes that are not often found in word-memory tasks commonly used 
to test Craik's and Lockhart's (1972) depth-of-processing concept. We can 
see some parallels between ^hese two areas. Crajk and Tulving (1975) 
found that subjects do not/ remember "... what was J out there' but 
rather what they.^aid durijfig encoding^" We. predict that aided problem- 
solvers will remember best (and perhaps enjoy most) the difficult but ^ 
productive^nfer^nces. An6t^'er poipt of^^•poss^b^e agreement with the 
depth-of-processing idea "Concerns the "number^ of features checked," 
As^suming some analogy with/the problem-solving case, a deeper inference 
.is one requiring recognition, selection, and scanning, of several phrases 
across several sentences/ A membership problem with several variables 

will elucidate .the point./ 

* • 

The five events in'the annual Boys' High intramural swinming meet— one 
Was a butterfly race— were won by five different "Animal League" teams, 
which then competed agaijist one another to determine the" teams' overall 
ranking. From the following clues, can you find the event each t^am ' 
won, the name of its.captain (one was Ned), and the final ranking of ( 
the teams? 
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1. Will, was not the captain of the backs troKe winners or the 
diving champions. . 

2. The Bears did not win the freestyle race. 

3.. The team that won the breastroke event finished ahead of 
the Leopards,- but behind both'WilTs team and the one that 
won the freestyle event. 

4. Tom's team was not the Tigers or the Leopards. 

5. The Bears finished ahead of the Lions. 

6.. The Panthers did not win the breastroke event, nor did \ ^ 
the Leopards trijjmph in diving, ' 

7. The Panthers did not finish last, but they were behind 
Paul's team.- 

\ — 

8'. The backstroke winners afi^.,^tM Tigets^ and Steve's team 
all finished behind the Lion^ ' . ^ 



Within ten minutes, many adul ts^^attempting^his problem will see 

that the Bears were in first place, and the Lions second; also, since 

Will's- team is not either backstroke, diving (clue 1),, -breastroke, or 

freesty.le (clue 3), Will's team has to be the butterfly swimmers.^ If. 

is easy to peg the Leopards; too; the .Leopards cannot be breastroke, 

freestyle, or butterfly (clue 3); and they Cannot be the diving team 

(clue 6); so- they must be the backstroke team, and- they $ilso can have^ 

fi-nished'no higher than fourth (clue 3). We now have a good start on 

the problem; to finish it, we will probably have to realize that Tom and 

i 

Will are on the top two teams; 'and only a few solvers will realize this, 
evep if given half an hour or more to work on the problem (some subjects 
may eliminate some of the possibilities^ ^'permute" the rest, and thus 
reach a correct assignment without going through all the 60 assignment 
possibilities; when they proceed nn this way, they would be imitating ^ 
the Fn?ST program).' The psychological difficulty is, that, to infer the 
Tom-Will placement, a lot of preliminary information has to be developed; 
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and then a rank-ond'er-rof-finish table processor has to operate simuT- 
• taneously on sev%^§i1^its of \/erbal data. .{From (8), Steve jnust be 
the leader of th6^anthersi the Panthers must have finished fourth; 

r 

and also from {7)v Paul's Tigers must be third). Thus, there are many 
aspects to "hold'^Sat the same time, and these must'^b^ appreciated firmly 
•enough to be converted into computer-acceptable statements. As far as 
the computer can tell, one "CON" or "NOf-cbN'' assertion is as "good as ' 
another, but the datfei demands on the human for realizing the different 
relations are usualfiy quite disparate. ' i/ * - 

How might depthrof-processing concepts be .used in teaching' people 
to be good problem-solvers? One possibil-j^ty is to teach problems with 
easier or "shallower" inferences first, up to a strict performance cri- 
terion, and then gradually to increase^.*h^epth of .i-nfefences via . 
controlled practice. A program of thns sort mivght be d,esigned to be 
adaptive, in the sense it would adjust the practice to the "best;expected 
gain" per unit time at the terminal. There' are several empirical matters 
to investigate: the bases for ojj/ljering the problems in the training set, 
, expected transfer effects across problems, the proportion of variance due 
to aptitude or knowleSS^e-jdifferences, the extent to which processing tricks 
and gimmicks can be taught, and so ort. 

Another project could focus on the extremely "deep" -or.clifficul t in- 
ferences. Here the research strategy wotild be that, if the subject's per- 

✓ 

formance on these hardest parts could be iir^proved, then the easier t^isks 

would take care of themselves. To teach j:he deejier inferences, specific ^; 

^' training analyses would be done for each diffic/lt inference, and the^7 

student would N^alk through these examples. 

f ' ' 

\ 
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5. Keeping Score . We noticed that, when using the FIRST 
routine, a student will often tend to .make rather too many "status 
9lieT:ks;" that is, he/she will frequently ask for a printDut^"to see 
if I've solved it yet." This feature was provided in the program as^ . 
an informatiorfe^/l aid, but income cases it may actually^ serve to ob- 
scure the logical prrocess. Perhaps the -student gets involved with 
"getting Jthe answer," and is visibly disappointed or exhilarated when 
' the table is filled in. This makes it more of a game, all right, but 
doer, not necessarily instruct the players!^ Perhaps future program 
versions should not permit so many table checks. - 

»6. Differences between Analyzing Logic Tables and Human Inference 

{ [ ;^ ' ' 

Behylor . Our programs that operate upon decision tables a^e necessarily 

"clean," with nice I's^and O's in the cells. Also, there are definite * 

- . ^ . 

evaluation rules in the program lyhich decide whether the problem^is solved 

^ or not; the processing is aimed directly at getting , a clear resolution of 

the set-membership- relations. Actually, of course, human inference be- 




^havior is' often far less than^ certain, and it may nbtkhow'just "where *it 
' % ' \ ^' ^ 

, is going." /As Schank (1975) put itJ ■ • 

" the (real) process of generating. conceptual 

inferences is inherently a computationally wasteful 
process, because its tntent is- to discover what, is 
interesting in A particular context." , 

This means that we should Expect much elaboration ^^havi^pr as^ubjects 

work on a problem. , It/ may be possible, . thVjough directed prac\ice, to 

facilitate a certain "directness" in, the elaborative activities of sub- 

jects. Certainly many verbal puzzles have common dimensions; aften'the 

problem-rests on variables like age, parent-child kinship relations, the ' 

days of the week,|^ank plac"eift®Tr^~s6me criterion, such as, money of" ^ 

. ^ ' ) • 
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Other coQntable outcomes. Or physical contiguity of events. Suppose 
that these standard dimensions could tfe listed, and specific elattora- 

4 

tive operations be planned for each dimension. ^ Then it'shoulj be a ^ 
direct task to teach the accessary elaboratiye behaviors in a set of . 
problems; perhaps a computerized scratch-pad could be provided for each ^ 
of the candidate. dimensions. ' ° ^ < 



When a neutral observer watches a problem attempt, a" freqyent 
occurrence is that the solver ^will "graze," but still "miss'," -a key 



implication of a statement. In at least some cases, the trouble appears 
to be that a scan of a statement}^ or of two or more statements, alternate 
between two rather different processes: (1) disQOVering wh?Lt the dimen- 
sion should be, and (2) evaluating the statement for any new inferences 
that may come from the dimension. PerhafiDS these aspects should b^' arti- 
ficially separated, at least in a training pr'ogram. - , 



V. IMPROVEMENTS AND EXTENSIONS 



While tryouts have shown the feasibility of decision-table soft-- 
«are*in verbal problems, there ha^ been no thorough evaluation. of the 
programs as'teaching aids. Before such an evaluation is undertaken, ^ 

» 

the programs need more fj^atures and capabilities than they have now. 
Some^ajoV changes planned are listed itl the paragraphs below; in'this 
material, we have limited consi'xleratipn to those items that seem possible 
^ri^ht now^ 4 ^ ^ ' • ■ . ' ^ 



1. Rank-Order Dime nsional Store^ ^ A problem-solver often needs 



to put his membership variables in some^ order. In age-related problems, 
mothers and fathers are older -.than sons and daughters; in the Swimmer 
problem on pages' 23 and 24, you prjjfjy will never get the answer unless 
you see that the Bears and Lions are the' top two^eams, and that ^he 
Leopards are on the bottom with Paul's and Steve's teams in between. 
Such' processing is done as an; intermediate step. ^ software ai^d should 
have a call up feature tfiat permits order information to be collected and 
storei ^outside the usual CON, NOT-CON, and conditiona^^J^es. Probably- 
three rank-order .dimensions would be sufficien^t for most problems. The 
solver could define and use these as "working files" while he is combin-^ 
ing information, from two or more sentences; once he hasr a.^firm membership 
statement he can go to his regula/ CON table entry. 

Here's how it might work. Returning to* the Swimmer problem, a rank 
order file might be defined as "order of finish, with five sjots, 1-5." 
From premise 5, the soVver would enter "Bears ahead of' Lions;" from 
premise 8, "backstroke team," Tigers, and Steve's team behind Lions," 
The system now knows that Bears and Lions are first and second. If the 
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solver looks at premise'7, he sees that the last team cannot be the ^ 

*• . . ^ \ ~ 

Panthers, so he enters "Panthers not last" or some equivalent. There 

are only four possible orders remaining: 

>^ BEARS . BEARS BEARS _ BEARS 

LIONS LIONS LIONS > LIONS 

-TIGERS PANTHERS 1.E0PARDS PANTHERS 

. . PANTHERS TIGERS PANTHERS ' LEOPARDS 

LEOPARDS LEOPARDS TIGERS TIGERS ; 

■ V - - ' • ■ \ 

Seeing^this -table, the solver now may focus^^ further search to re- 
solving the .third-fourth issue for the Panthers, or' perhaps to the place=" 

tKKLeopards. * , , . 
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)2. Stoning Problems . It is a nuisance to start .each pro^b^em with 
a separ'ate^pi4ce .of paper; ihis necessity also retires an attendant to 
stand around wlilile the solver is working. Future versions of FIRST wijl 
allow for storage ^f a'dozen or so problems; before a session begins, ^he 
attendant will se^:^in the order &f problems', and then leave the solver \^ - 
alone. With new memories offering a quarter-of-a-million words, of storage^ 
there should be vpo further need for manual problem startsl Another sofS * 
ware addition will be a problem restar^procedure- which will be easy for 
the subject to use. * < - 

3. Scoring System for Depth of Inference . As a silent accompani- 
ment to the student's work, subroi^ines will be installed to figure con- 
titftious "depth-of-inference" scores. First attempts at doing this will 
use simple probabili"ty-of-success indfcatprs^.for each cell in the matrix, 
including whether^or>ribt tbe entry was achieved^^^ inclusion ©r^exc(Iusion 
logic. There will also" be rough (1-minute -increment) time scores for each 
logic entry. Every CON, NOT-CON, and conditional entry into a basic prob- 
lem matrix will be flagged for this scoring system. ^ 



^agg^c 
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Several groups of subjects have been asked to recortstruct their 
logic, ininediately after working on such problems as the Murderer. While 
the main results of those studies will be ^iven in another BTL report, 
we can mention here that, for some problems, it is quite feasible to 
determine just whith logical path a given sutiject followed/ This is ^ 
possible bei:ause there are or^ly a' few paths to a (log.ica1) solution. .The 
Murderer has four paths, and one of ^hese is by far tb/^most popular. 

It seems .that a scoring system-might try to t:rack the logical path 

of each subject pn each problem, and print outa^final Recount of just 

where the subject got as he worked on the problem. This might be a digger 

» 

^software job than it appears to be right now. We expect to explore it 
first with a few problems wherein we already know all the logically admiss- 
able paths, and' where we have some idea of the success probabilities at 
each node in* the path. 

Automatic display of the logical^ath aclTTeVed by'a^subject might be 
a helpful t^e€fching aid in itself. Suppose that a subject has completed 
aM but one tfr two iTiferences in a path; the displjiy might be a good way 
for him to review his performance. A major challenge here to the software 
designer will be to provide a useful, but not overly complex, printout. 
For instance, should little remediation se/itences, elements, and advices 
be put on the logical-path review, at those points^where the solver missed 
something?:^ * * ' <, 

4. Intersentence Processing . I-f the critical, relations in a ' 
pVoblem flow from the combination of data from several sentences, then " 
a ^ftware aid should do something definite about this part of the solution 
attempt. ^So far, we can formulate several heuristics which might be 
generally useful. The^irst of these would urge the solver to ask. for a 
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status printout after his first descriptive pafss through the sent^ces, 
'and then toflook closely at^those variables' which appear ^be the o 
nearest to being locked up, or totally defined jnTTroblem terms. Then 

t 

these J&rticular variables are 'scanned across sentences, to see if any 
more CON or NOT-CON relations can be found. 

A second heuristic would recommend that, once a solid CON is. achieved 
' in the problem table, the possibility of further NOT-CON's can be made by 

0 - ' . • ^ 

rereading pairs of sentenced contaf^ining the element which has gust been 
"CON'ed." . • ' * 

• 9 

As a third technique, thd most infornlative sentences are apt to be 

.those with a lot of words 'and fe^clusions in them. Taking tv(p of these 

"high^ information statements together might be a good thii^ to do, if a 

solver is temporarily stuck. .Sometimes, too, a key sentence will ha:Ve 

' ' . \ 

data on two or more dimensions in it; in^the Swimmer problem, premise 3 
yrseparates VJill and LeOpards from three other probl em-el emen*^ and also 
gives the indication that the Leop^ards cannot be befter than fourth. In 
fact, about nine definite logical statements can be obtained from that 

X 

one premise. '» . ) • 

It is a question whether such'^heuristic/ can be suitably defined 
over a bro^d problem set;' and-therfe is a further <westion whether such 
heuristics can be utilized to advantage in new problems.. We are ^^imis- 
tic at the moment^ partly because hfi^^rtstics are eminently, teachable^ in 
other, logical, domains (such as .^setting up integration problems in ca-lculus), 
and 'partly because although the words 'in verbal problems are complex, Jhey 
, aren^t so complex that .most terms cannot be dimensionally analyzed. Even 
a partial system for rolling over the dimensions may be enougVto promote 

a key inference. - . ' ' • 
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•On long and iavolved problems like the Swimmers, the solution is 
bound to take some minutes, and there is an interesting' point when the 
solver begins to think that he/she bats just about broken- fcHe'^^obl em, 
and that everything will soon fall into place. Sometimes it can even 
happen that the solver Already has enough logic to fill in the answers, 
if the information is just collected from all the tabular arrays. A 
small aid here migbi be a computer^ subroutine which would provide a 
running "logi^ score;" when this score is, s,ay, between 0 and 1 , then 
the solver should continue to derive new logical inclusions and ex- 



^clusions. When the score goes over 1.00, then the solver knows that 
- he can>^sily solve for remaining unknowns, with the inferences he has ' 
already achieved. Thus, if" your score, is 1.08, then your main task is 
t£b collect, from the several arrays and tables, all the facts you now 
, have. As yet, there seems^to be no corrpletely general way to do this 
^ cal€ulationi_biLt it can certainly be' prograijmed for^each problem sep- 

arately./ It v/ould certainly be a shame for a solver to ha'Ve.^nough data, 

' ^ . V ' ' ' 

and nol^ If now i t! ^ • ' ^ 

i 5. Automatic Composition of Logic Tables . Experienced^ problem- 

solvers may prefer to set up their own ]ogik. tables, trees, and other 
. bookkeeping devices; the authors, for instance ^ often find themselves, 
scribbling little bits of ordering data oriexclusion logic-, when working 
on a vertiaU-problem. , These notes are usually incomplete and. rather hit- 
and-run; as in the Schank ipjote earlier, we^ are looking for something' 
that is logically interesting. We believe, however, that most subj-ects 
like to have the computer provide to them a clear (empty) table to start 
with. In the Swimmer, there would be four main dimensions (team name. 



4 



place. Captain's name, style of stroke) with five rows or columns on 
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each dimension* Future runs of the FIRST. program- will immediately print 
out a table like this, and encourage the subject to tear it off and' use 
it as a starter recording device* At any time, ,the program will also be 
Capable of printing out an up-to-date marked version, if the instructional 
circumstances demand it. 

5. Time and Rate Indexes > Several investigators have postulated 
that individuals ^differ radically in their basic information processing 
capacities. Hunt (1977), for example, was able to rank-order several , 
groups of people according to their response latencies in some simple dis- 
crimination tasks, A computer-aided system operating on logical material 
should be able to yield a similar "basic inference rate" over a series of 
Standard sentences, and to tabulate t-his for each subject. In the next 
series of trials, we plan to explore this possibility in some dq^tail. Of 
special interest here will be the correlation of performance on single- 
sentence logical processing, with. a score on inter-sentence deri vatiojis. 
We will also be lookirig at the parametric and distributional features of 
rate measures, in this^ domain, just as Hunt examined intercept and slope 

V 

features of his speed measures. It is probably over-optimistic to think 

that one or two basic logical -processing parameters can really describe 

1 

performance J n difficult verbal prpblems'^ bjjt^it is reasonable to think 
that they can tell more about the processes than most other kinds of 
predictors. 



9 ' 
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